AMENDMENTS TO THE SPECIFICATION 



Please insert, after paragraph [0007], the following new paragraphs. 

[0007a] FIG. 3 shows Hot Swap Event Channel Initialization Sequence 
[0007b] FIG. 4 shows Asynchronous Event Distribution Sequence 
[0007b] FIG. 5 shows Synchronous Pre_Extraction Event Distribution 
Sequence 

Please amend the following three paragraphs as follows. 

[0057] FIG. 1 is an example of a conventional CompactPCI Hot Swap 
implementation, being page 140 , FIG. 30 in Appendix A of CompactPCI Hot j 
Swap Specification PICMG 2.1 R.20 (Jan. 17, 2001). FIG. 2 is a circuit diagram 
that, according to the present invention, allows a board to satisfy both the 
CompactPCI Hot Swap Specification (with examplary reference to FIG. 1) and to 
detect the opening of its own handle. Attached hereto, and incorporated as part 
of this application, is draft CompactPCI Hot Swap Specification PICMG 2.1 D0.91 
(Feb. 5, 1998), which is equivalent to CompactPCI Hot Swap Specification 
PICMG 2.1 R.20 (Jan. 17, 2001) referred to herein for purposes of this invention 
(i.e. the differences between the draft and the standard are not relevant to this 
invention). 

[0069] .Amplification of the CORBA implementation aspects of this invention is 
set out next found be l ow at th e end of th i s spec i f i oat i on. as pages 10a to 10g with 
drawings on pagos 10h to 10j, which ar o incorporat e d horoin ao part of th is 
specificat i on . 

[0070] A l though tho method and apparatus of tho prosont invention has boon 
descr i bed in connoct i on w i th the preferred embod i m e nt, i t i s not i ntended to bo 
l imited to tho specif i c form s o t forth horoin, but on th o contrary, it is intended to 
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cov o r such alternat i ves, modifications, and equiva l ents, as can bo r e asonably 
i ncluded w i th i n th o sp i r i t and soopo of the invent i on as defined by tho appended 
etajmsr Distributed Hot Swap Management in an Embedded System 

Please insert, after paragraph [0070], the following new paragraphs. 

[0071] Abstract. Many modern embedded systems have a need for minimal 
downtime due to failures, maintenance or upgrades, which is addressed by 
permitting insertion and removal of boards while the system is active. Both 
CompactPCI and VME systems support live insertion and/or removal of boards, 
in CompactPCI, this is termed "Hot Swap". It is important that notification of such 
a change in configuration is communicated appropriately throughout the system 
so that it can react appropriately, perhaps by redistributing work between the 
available resources or shutting down any direct communication between boards 
before the boards involved are physically removed from the system. Ina 
CompactPCI system, the notification takes the form of an interrupt to a System 
Host which takes the appropriate action. However, not only does the System 
Host constitute a single point of failure, but in many systems where there is no 
other traffic over the CompactPCI bus, handling this interrupt may be the only 
reason for the System Host to be present at all. Elimination of this centralized 
control function would therefore lead to a cheaper, more robust and optimal 
system architecture. 

[0072] In an embedded multiprocessor system, entities which need to know 
about insertion or removal events may be distributed throughout the system. 
Such entities may care about Hot Swap events in one of two ways: They may be 
interested in knowing about a Hot Swap event within some defined time of it 
having occurred, or they may need to know about an impending Hot Swap event 
and take some action before it occurs. In the former case asynchronous 
notification of completion of the event is sufficient, however the latter case 
demands synchronous action on the part of the interested entity. The mechanism 
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used for both cases should ideally provide complete decoupling of interested 
entities and sources of Hot Swap events and achieve an acceptable response 
time so that completion and notification of events is not unduly delayed. 

[0073] Below discusses how a modified version of the Real-Time CORBA 
Notification Service may be used to meet these requirements. 

[0074] CompactPCI Hot Swap. CompactPCI Hot Swap, as defined in the Hot 
Swap Specification [1], introduces the ability to insert or remove a Peripheral 
Board (any board other than the System Host) from a running system. Removal 
is a two-phase process. The first phase occurs when the lower latch on the board 
is opened, this generates an interrupt notifying the System Host that removal of 
the board is imminent. The Host then has an opportunity to quiesce the board - 
before initiating the second phase by lighting a blue LED . on the board, which 
indicates that it is now safe to fully extract the board. Hot Swap thus increases . 
the availability of CompactPGI systems by allowing the removal and replacement 
of Peripheral Boards without the need to power down the entire chassis. 
Unfortunately, the System Host is still a single point of failure. If the System Host 
fails or needs to be upgraded, the entire chassis must still be powered down in 
order for it to be removed and replaced. 

[0075] The Hot Swap specification imposes certain requirements on the 
software running on the System Host. In particular, on detecting that a Peripheral 
Board is about to be removed, the System Host is required to ensure that the 
board being extracted no longer accesses the PCI bus and that the board is not 
accessed from the PCI bus. This is necessary to ensure the integrity of signals 
on the bus as the board is removed. This requirement is reasonably easy to meet 
when all communication is between the Peripheral Board and the System Host, 
but significantly harder if there is direct communication between Peripheral 
Boards. 
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[0076] An Alternative Approach. In many CompactPCI systems, Peripheral 
Boards have a reasonable amount of intelligence. They may have fully functional 
CPUs and even run Operating Systems. For these boards, it is natural that they 
communicate directly with one another. 

[0077] In systems with a large amount of data to process in a short period of 
time, the bandwidth of the CompactPCI bus may not be sufficient. This is 
exacerbated by the fact that the available bandwidth is shared between all the 
Peripheral Boards and the System Host. 

[0078] In such a system, it may make sense for Peripheral Boards to 
communicate directly via some other medium, quite possibly in a point-to-point 
manner. This is the impetus behind the PICMG 2.16 [2], the forthcoming PICMG •■ 
' 2.1 8 [3] and other related standards, which introduce alternative communication 
links between Peripheral Boards. In systems such as these, the CompactPCI 
backplane may be little more than a source of power and a clock. A clock signal 
can easily be generated on the Peripheral Boards if it is not present on the • 
backplane but is needed for a local PCI bus on the board. This leaves the non- 
redundant System Host with just one job - supporting Hot Swap of the Peripheral 
Boards. 

[0079] Removing the System Host. If the work involved in handling Hot Swap 
events could be distributed to the Peripheral Boards, the System Host could be 
removed completely, eliminating the single point of failure. This turns out to be 
relatively simple to achieve. 

[0080] In a CompactPCI Hot Swap system, Peripheral Boards have a micro- 
switch attached to the lower board latch. When the latch changes state between 
open and closed, the System Host is interrupted and can deal with the event. A 
Hot Swap Control and Status Register in configuration space of the device is 
used to communicate the details of the event. With a CPU on the Peripheral 
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Board, it is relatively simple to implement similar functionality entirely on the 
board itself. Opening or closing the latch generates an interrupt to the CPU on 
the Peripheral Board. That CPU can query the hardware to determine exactly 
what happened and how to respond. It is even possible to implement a register 
that behaves exactly like a Hot Swap Control and Status Register, in which case 
the software that runs on the local CPU to handle Hot Swap events may be 
exactly the same software that usually runs on the System Host. This allows 
each Peripheral Board to independently detect that it is about to be removed. 

[0081] In a system with no System Host, there can be no communication over 
the CompactPCI backplane, so there is nothing there that needs stopping. There 
may of course be other communication links that need to be moved to a 
quiescent state. Surprisingly, there is a need to handle insertion as well as 
extraction. An insertion may look exactly like a power-up - the board moves from 
an un-powered state to one in which it has power. There is also the possibility 
that the latch was.opened, causing the extraction processing to be performed, < , 
and then re-closed without actually extracting the board and causing it to power- 
cycle. This change in state needs to be handled in much the same way as when 
the board first gets powered up. 

[0082] It is possible to design a board that detects whether there is a clock on 
the CompactPCI bus. If it detects such a clock, it should meet all the normal 
CompactPCI requirements. If it does not detect such a clock, it can assume that 
the CompactPCI backplane is not being used for communication and that there is 
no System Host present. In this case, the local CPU can take over the handling 
of Hot Swap management tasks as described previously. 

[0083] Types of Interaction with Hot Swap Events. In an embedded system with 
no centralized control, entities which care about insertion or removal events (Hot 
Swap event consumers), may be distributed on different, heterogeneous 
processors throughout the system. 
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[0084] Such consumers may care about Hot Swap events in one of two ways. 
They may merely need to react to a Hot Swap event after it has happened. For 
example different consumers may need to: 

• Initialize hardware that has been inserted. 

• Instantiate proxies for hardware that has been inserted. 

• Update the system state on a user interface. 

[0085] Although an asynchronous mechanism is sufficient, in fact desirable, for 
these types of notifications, it still needs to be efficient enough to achieve 
acceptable response times. 

[0086] Alternatively they may need to be directly involved in an impending Hot 
Swap extraction event in order to perform some processing before the extraction 
actually happens. For example different consumers may need to: 

• Quiesce themselves and save their state if they are physically located 
on the hardware to be removed. 

• Shutdown communications if they have a direct connection to the 
hardware to be removed. 

• Reconfigure the application if they are acting in some type of system 
supervisory role. 

[0087] Clearly a synchronous mechanism is required to notify all directly 
involved entities in a timely manner and ensure that they have completed their 
processing before extraction is allowed to proceed. 



[0088] General Requirements for Decentralized Hot Swap Event Notification 
Thus in order to support decentralized Hot Swap in a heterogeneous computing 
environment, we require an event notification mechanism that: 
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• Provides both efficient synchronous and asynchronous notification services. 
(While not required, it is desirable that these services are accessed via a 
common interface) 

• Supports diverse target processors. 

• Supports location transparency so that events are distributed identically 
irrespective of whether they are local to the hardware in question, within the 
same chassis or somewhere else in an arbitrarily large system. 

• Provides loose (i.e. determined at run-time) coupling between sources and 
consumers of Hot Swap events (since system configuration is by definition 
changing on the fly). 

• Is standards-based in order to maximize re-use and portability. 

• Offers redundancy to remove sensitivity to individual failures. .j 

[0089] The Chosen Solution. A solution based on the Object Management 
Group's (OMG) CORBA Notification Seryice [4] is chosen for the following 
reasons: 

• It is based upon the Common Object Request Broker Architecture (CORBA) 
specification [5] which is a well supported middleware industry standard 
which offers an architecture, language, transport and vendor independent 
communication in a location transparent manner. 

• jt provides an efficient asynchronous event distribution mechanism which 
offers advanced quality of service (QoS) and filtering properties (which were 
not present in its predecessor the CORBA Event Service). The QoS 
properties are used to control reliability, priority and timeliness of event 
distribution, while the filtering mechanism makes efficient implementation 
possible by suppressing unnecessary distribution of all events to all 
consumers. 
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• It provides the required loose coupling by means of a publish/subscribe model 
via which sources and consumers register independently with a well-known 
Hot Swap Event Channel and thus require no direct reference to each other. 

• Provided the CORBA implementation supports it, the Hot Swap Event 
Channel may be implemented as a redundant object in accordance with the 
Fault Tolerant Specification [Chapter 23 of reference 5] . 

[0090] The Notification Service does not provide a synchronous event 
distribution but its interfaces and semantics may be extended to provide such a 
mechanism as discussed next. 

[0091] Implementing a Hot Swap Event Channel. During system initialization a 
single Notification Service event channel is instantiated. This Hot Swap Event 
Channel registers itself at a well know location within the CORBA Naming 
Service. 

[0092] Any consumer which wishes to be notified of any kind of Hot Swap event • 
locates the Hot Swap Event Channel by means of the Naming Service and 
registers itself to receive Hot Swap Events (see FIG. 3 - Hot Swap Event 
Channel Initialization Sequence). The filtering mechanism provided by the 
Notification Service allows a consumer to specify only the exact kinds (Insertion, 
lnsertion_Error, Pre_Extraction, Post_Extraction and Channel_Shutdown) and 
sources of Hot Swap events that it cares about. This eliminates the overhead of 
the service distributing and the consumer ignoring events which are of no interest 
to it. 

[0093] For example, a consumer that needs to perform some processing before 
a particular piece of hardware is extracted, needs to register for the 
Pre_Extraction event from that particular source, while a consumer that needs to 
be notified of system configuration changes would register for the Insertion and 
Post_Extraction events from all sources. The InsertionJError event is generated 
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when hardware is inserted but is unusable for some reason (for example a self- 
test failure). Some consumers might register for all events. The consumer will be 
called back by the Hot Swap Event Channel ("Channel") when an event in which 
it is interested (i.e. matching its filter) occurs. 

[0094] Sources of Hot Swap events are typically service routines responding to 
the Hot Swap interrupt on processors residing on removable hardware, however 
the System Host in a CompactPCI chassis which contains only dumb peripherals 
may also be a source distributing events to a wider Super-System. It is also 
possible for a device (such as a system health monitor) in the system to inject 
Hot Swap events for boards on which it detects a failure. When a Hot Swap 
source has an event to distribute, it locates the Hot Swap Event Channel in the 
Naming Service and calls the Channel with the event. 

[0095] Thus a push model for event distribution is employed where sources 
push events to the Channel, which in turn pushes the events to interested 
consumers. Structured events as defined by the Notification Service are used 
since they provide filtering support. 

[0096] Asynchronous Event Distribution. Asynchronous distribution is used 
whenever a source has an Insertion, lnsertion_Error, Post_Extraction or 
ChanneLShutdown to report. This is the normal "fire and forget" mode of 
operation of the Notification Service in which the push call from the source 
returns immediately, the Channel thereafter passes the event through the 
applicable filters and distributes it to all interested, registered consumers (see 
FIG. 4 - Asynchronous Event Distribution Sequence). In an efficient 
implementation, multiple threads are used to notify interested consumers 
simultaneously [6]. 

[0097] Synchronous Event Distribution. In order to support synchronous 
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participation in Pre_Extraction events, an extension to the Notification Service is 
needed to provide synchronous behaviour. The semantics of this extension 
otherwise match the standard Notification Sen/ice specification. 

[0098] A synchronous event is generated when a source calls the push 
operation on a newly defined synchronous interface on the extended Notification 
Service. This event is distributed in exactly the same way as an asynchronous 
event, except that the source is called back when all consumers have completed 
their processing (or otherwise failed or timed-out) with an indication of the 
success or reason for failure of the total operation (see Fig. 5 - Synchronous 
Pre_Extraction Event Distribution Sequence). This permits the source to wait for 
completion of the event distribution before continuing or alternatively implement 
its own timeout if no response is received. 

[0099] Extraction Processing. Thus each extraction which is initiated, first < 
generates a synchronous Pre_Extraction event. notification to ensure that all 
required pre-extraction processing is complete, followed by an asynchronous 
Post_Extraction event notification prior to lighting the blue LED for extraction to 
complete. 

[0100] Fault Tolerance. One of the stated advantages of eliminating the need 
for a System Host from a CompactPCI bus was the removal of a single point of 
failure from the system, however when using the extended Notification Service all 
events must pass through a Hot Swap Event Channel which must reside on a 
single processor which again becomes a single point of failure. 

[0101] This problem is addressed by the Fault Tolerant CORBA specification by 
providing for distributed redundant replicas of CORBA server objects and 
mechanisms to switch between these replicas to deal with failures. This is 
completely transparent to users of the service. Therefore, if required, the Hot 
Swap Event Channel can be implemented as a Fault Tolerant object to provide 
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additional system reliability without impacting Hot Swap event sources and 
consumers. The Hot Swap Event Channel therefore meets all the above 
requirements for a decentralized Hot Swap control mechanism. 

[0102] Summary. With the aid of appropriately configured middleware, it is 
possible to distribute Hot Swap control among intelligent computing devices in a 
CompactPCI chassis thereby eliminating the need to centralize this function on a 
System Host. Moreover events may be distributed throughout a Super-System 
consisting of multiple chassis both with and without System Hosts. This has both 
reliability and system management benefits. 

[0103] An enhanced implementation of the CORBA Notification Service 
provides a suitable middleware platform to support distributed Hot Swap.control. 
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[0105] Although the method and apparatus of the present invention has been 
described in connection with the preferred embodiment, it is not intended to be 
limited to the specific form set forth herein, but on the contrary, it is intended to 
cover such alternatives, modifications, and equivalents, as can be reasonably 
included within the spirit and scope of the invention as defined by the appended 
claims. 
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